Of image processing techniques, a technique for reducing noise included in an image is a necessary technique in order to reproduce a captured image more clearly. As one of the noise included in an image, there is additive white Gaussian noise as illustrated in FIG. 10. An original signal and a signal deteriorated by noise are designated as s and f, respectively. At that time, assuming that an amplitude of a noise nG follows a Gaussian distribution, the additive white Gaussian noise of FIG. 10 is represented by f=s+nG.
As a technique for reducing noise included an image, for example, a Total Variation norm minimization method (hereinafter, expressed as a TV method) that is a technique disclosed in NPL 1 is widely used.
The TV norm is obtained by adding absolute values of differences between adjacent signals in the whole signal section, and a TV norm of a signal u is defined by the following equation (1):TV(u)=∫Ω|∇u|dx  (1)wherein ∇ represents a partial differential operator.
FIG. 11 to FIG. 13 illustrate examples of the TV norm with respect to a one-dimensional signal. A signal (flat signal) gently changing illustrated in FIG. 11 and a signal including a steep edge illustrated in FIG. 12 have the same TV norm. As illustrated in FIG. 13, when a signal value vibrates, the TV norm has a large value. In other words, the TV norm can be referred to as an index for indicating a smoothness of a signal in consideration of discontinuous components.
The TV method is a method for separating an input image f including noise into a skeleton component u including an edge component and a flat component of the image and a residual component v including a texture component and a noise component. In other words, the relation of the following equation (2) is established.f=u+v  (2)
The skeleton component u can be determined by minimizing equation (3) in which a normalization term is introduced into the total variation norm TV (u) defined by the equation (1).
                              T          ⁢                                          ⁢                      V            ⁡                          (              u              )                                      +                              μ            2                    ⁢                                    ∫              Ω                                                                    ⁢                                                            (                                      u                    -                                          u                                              (                        0                        )                                                                              )                                2                            ⁢                                                          ⁢                              ⅆ                x                                                                        (        3        )            
In equation (3), u(0)=f is satisfied and μ is a parameter for indicating a fidelity of an original image.
Equation (3) is described below with reference to FIG. 14. A signal illustrated with a solid line in FIG. 14 is designated as an input signal f including noise. TV norms of a signal u1 illustrated with a dashed line in FIG. 14 and a signal u2 illustrated with a dotted line in FIG. 14 each are 10 and are equal to each other. However, as the original signal f and a skeleton component u separate from each other, the second term of equation (3) becomes larger, and therefore in order to minimize above equation (3), a signal illustrated with the dotted line closer to the original signal f is calculated as the skeleton component u.
As described above, the TV method causes to separate an input image including noise into a skeleton component (TV norm: small) including an edge and a flat area of the image and a residual component (TV norm: large) including texture and noise. This skeleton component can be regarded as a noise-eliminated image.
FIG. 15 illustrates a block diagram of one example of an image processing method related to the present invention that separates an input image into a skeleton component and a residual component.
The related image processing method includes a component separation means 11.
An input image f is input to the component separation means 11, and a skeleton component u and a residual component v are output.
FIG. 16 illustrates a block diagram for illustrating the component separation means 11.
The component separation means 11 includes a skeleton component extraction means 21 and a residual component calculation means 31.
The skeleton component extraction means 21 receives an input signal f and minimizes equation (3) to obtain a skeleton component u. The obtained skeleton component u is output to the residual component calculation means 31.
The residual component calculation means 31 receives the input image f and the skeleton component u output from the skeleton component extraction means 21 and determines a residual component v from the relation v=f−u based on the relation of equation (2).
As a method for extracting a skeleton component by minimizing equation (3) in the skeleton component extraction means 21, there is Digital TV Filter (DTVF) that is a technique disclosed in NPL 2.
FIG. 17 illustrates a block diagram for illustrating the skeleton component extraction means 21 that extracts a skeleton component in accordance with DTVF, and operations of respective processing units are described below using the flowchart of FIG. 18.
In the following description, a pixel value in a pixel location α of an image u is expressed as uα. A set of neighbor pixel locations of α is expressed as N(α). FIG. 19 illustrates examples of neighbor pixel locations of a pixel location α=(i,j). Herein, pattern (a) of FIG. 19 is referred to as a four-neighbor matter and N(α) is expressed as N(α)={(i,j−1),(i,j+1),(i−1,j),(i+1,j)}. Further, pattern (b) of FIG. 19 is referred to as an eight-neighbor matter and N(α) is expressed as N(α)={(i,j−1),(i,j+1),(i−1,j),(i+1,j),(i−1,j−1),(i−1,j+1),(i+1,j−1),(i+1,j+1)}.
The skeleton component extraction means 21 includes a local variation amount calculation means 101, a filter weight calculation means 102, a filter coefficient calculation means 103, and a filtering execution means 104.
Initially, as an initial setting, a counter variable k for storing the number of repetitions is set as k=0. An input image f is designated as an initial skeleton image u(0) (step S1).
The local variation amount calculation means 101 receives an image u(k) and calculates a local variation amount |∇αu| based on the following equation (4) (step S2). The local variation amount |∇αu| obtained by the calculation is output to the filter weight calculation means 102.|∇αu|=|∇αu|ε=√{square root over (ΣβεN(α)(uβ(k)−uα(k))2+ε2)}  (4)
ε in equation (4) is a parameter for preventing division by 0 in the following processing and may be a value of about 10−4.
The filter weight calculation means 102 receives the local variation amount |∇αu| from the local variation amount calculation means 101 and calculates a filter weight wαβ(u) represented by the following equation (5) (step S3). The filter weight wαβ(u) obtained by the calculation is output to the filter coefficient calculation means 103.
                                          w            αβ                    ⁡                      (            u            )                          =                              1                                                                          ∇                  α                                ⁢                u                                                            +                      1                                                                          ∇                  β                                ⁢                u                                                                                      (        5        )            
The filter coefficient calculation means 103 receives the filter weight wαβ(u) from the filter weight calculation means 102 and calculates filter coefficients hαα and hαβ based on the following equations (6) and (7), respectively (step S4). The filter coefficients hαα and hαβ obtained by the calculation are output to the filtering execution means 103.
                              h          αα                =                  μ                      μ            +                                          ∑                                  γ                  ∈                                      N                    ⁡                                          (                      α                      )                                                                                  ⁢                                                          ⁢                                                w                  αγ                                ⁡                                  (                                      u                                          (                      k                      )                                                        )                                                                                        (        6        )                                          h          αβ                =                                            w              αβ                        ⁡                          (                              u                                  (                  k                  )                                            )                                            μ            +                                          ∑                                  γ                  ∈                                      N                    ⁡                                          (                      α                      )                                                                                  ⁢                                                          ⁢                                                w                  αγ                                ⁡                                  (                                      u                                          (                      k                      )                                                        )                                                                                        (        7        )            
The filtering execution means 104 receives the initial image u(0) and the image u(k) that is a filtering target, as well as the filter coefficients hαα and hαβ from the filter coefficient calculation means 103, calculates an image u(k+1) after filtering of k+1 times based on the following equation (8) (step S5), and increases the repetition number counter k by 1 (step S6).uα(k+1)=ΣβεN(α)hαβuβ(k)+hααuα(0)  (8)
Then, k and a maximum repetition number kmax are compared, and when k is less than the maximum repetition number kmax, for the next repetition calculation, an image after filtering is output to the local variation amount calculation means 101 and the filtering execution means 104 and the processing is continued. When k has reached the maximum repetition number kmax, the image is output as a skeleton component u (step S7).
As is clear from equations (5), (6), and (7), when a local variation amount is sufficiently large compared to a noise component, hαα≈1 is satisfied, and therefore a blur of an edge is preventable. In contrast, when the local variation amount is small, DTVF regards this area as being flat and hαα≈0 is satisfied, and therefore DTVF behaves like a common lowpass filter. Regarding μ, using an estimated noise standard deviation σ, the relation: μ=1/σ2 may be satisfied.
FIG. 20 illustrates a block diagram of a related image processing device 1000 that separates an input image into a skeleton component and a residual component.
The image processing device 1000 includes a local variation amount calculation unit 1001, a filter weight calculation unit 1002, a filter coefficient calculation unit 1003, a filtering execution unit 1004, a residual component calculation unit 1005, a control unit 1006, a switch 1007, a switch 1008, and a memory 1009.
An input image f is input from an input buffer to the switch 1007 and the memory 1009.
The control unit 1006 is configured using a microcomputer or the like and executes a stored program to control the respective units. The control unit 1006 includes a counter variable k that stores the number of times of filtering. When an image is input from the input buffer, the control unit 1006 resets the counter k to be k=0, switches the switch 1007 to output the input image f to the local variation amount calculation unit 1001 and the filtering execution unit 1004 as an initial skeleton image u(0), and switches the switch 1007. Then, the control unit 1006 outputs an image received not from the input buffer but from the switch 1008 to the local variation amount calculation unit 1001 and the filtering execution unit 1004.
The control unit 1006 monitors the filtering execution unit 1004 and increases the filtering number counter k by 1 when filtering is completed. Then, the control unit 1006 compares k and a maximum repetition number kmax, switches the switch 1008 so as to output an image to the switch 1007 when k is less than the maximum repetition number kmax, and switches the switch 1008 so as to output an image to an output buffer and the residual component calculation unit 1005 as a skeleton component u when k has reached the maximum repetition number kmax.
The switch 1007 and the switch 1008 are switched in accordance with control of the control unit 1006.
The memory 1009 stores the input image f.
The local variation amount calculation unit 1001 receives an image u(k) from the switch 1007 and calculates a local variation amount |∇αu| based on the above equation (4). The local variation amount |∇αu| obtained by the calculation is output to the filter weight calculation unit 1002.
The filter weight calculation unit 1002 receives the local variation amount |∇αu| from the local variation amount calculation unit 1001 and calculates a filter weight wαβ(u) represented by the above equation (5). The filter weight wαβ(u) obtained by the calculation is output to the filter coefficient calculation unit 1003.
The filter coefficient calculation unit 1003 receives the filter weight wαβ(u) from the filter coefficient calculation unit 1001 and calculates filter coefficients hαα and hαβ based on the above equations (6) and (7), respectively. The filter coefficients hαα and hαβ obtained by the calculation are output to the filtering execution unit 1004.
The filtering execution unit 1004 reads the input image u(0) from the memory 1009, receives the image u(k) as a filtering target from the switch 1007 and the filter coefficients hαα and hαβ from the filter coefficient calculation unit 1003, and calculates an image u(k+1) after filtering of k+1 times based on the above equation (8). The image u(k+1) after filtering of k+1 times obtained by the calculation is output to the switch 1008.
The residual component calculation unit 1005 receives the input image u(0) from the memory 1009 and the skeleton component u from the switch 1008, and obtains a residual component v from the relation of equation (2). The obtained residual component v is output to the output buffer.